Image processing apparatus having image region specifying function

ABSTRACT

A plurality of image regions in the image of the original document are determined on the basis of the image data of the original document. The positions of the plurality of image regions are then indicated as regions in the image of the original document by the display. With use of the display, the user specifies at least one point in the image of the original document in order to specify one region. On the basis of the coordinates of the position specified by the user and the relative positions of the image regions, the image region corresponding to the specified position is determined as the specified region. The image in the specified region is then subjected to the image process desired by the user.

TECHNICAL FIELD

This invention relates to an image forming apparatus such as a digitalcopying machine, in particular, a technique of specifying a region in animage and processing it.

BACKGROUND ART

In the conventional image processing, a scanner provided in a digitalcopying machine reads an image on an original document. A userdesignates a region of the image thus read, by specifying at least twocoordinates. The image designated is subjected to processing such asedition or insertion.

To designate the region of the image, the coordinates must be specifiedvery precisely, however. Hence, the user needs to spend much time andlabor to specify the coordinates if the region has a complicated shape.

In stead of specifying the coordinates of the image region, there isprovided another method by which a desired image region in an originaldocument is specified with a marker or the like in advance, and then theoriginal document is read by a scanner to recognize the specified imageregion. According to this method, however, it is very troublesome when aplurality of original documents are given for a user to manually specifyeach of image regions in each of the original documents.

DISCLOSURE OF INVENTION

The object of the present invention is to provide an image regionspecifying method by which a user can easily specify a desired imageregion on the basis of the image discrimination result of a readoriginal document (by one action, for example), an image processingapparatus for processing the image of the specified image region, and animage forming apparatus.

In order to solve the aforementioned problem, according to one aspect ofthe present invention, there is provided image processing apparatuscomprising: image region determination means for determining a pluralityof image regions in input image of the original document on the basis ofinput image data of the original document; display means for displayingpositions of the image regions determined by the image regiondetermination means in the image of the original document; specifyingmeans for specifying at least one position of a point on the basis ofcontents displayed by the display means; determination means fordetermining an image region corresponding to the specified position as aspecified region on the basis of coordinates of the position specifiedby the specifying means and relative positions of the image regions; andimage processing means for processing an image in the specified regiondetermined by the determination means.

Image region determination means has means for determining more than onenon-image region located between the determined plurality of imageregions. The determination means determine the non-image region as thespecified region when the specified position is located in the non-imageregion determined by the image region determination means. The displaymeans has means for displaying the specified region determined by thedetermination means so as to be discriminated from the other imageregions, for example, by changing the brightness therein.

The display means has means for displaying a contour of each of theimage regions determined by the image region determination means at acorresponding position in a frame of the image of the original document.

The image region determination means comprises means for calculating aposition and a size of each of objects constituting the image of theoriginal document in order to supply them as a featuring amount; andmeans for determining a plurality of image regions by unifying each ofcollections of the objects having the same attribute on the basis of thefeaturing amount.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing the constitution of an image formingapparatus according to an embodiment of the present invention;

FIGS. 2A to 2C show an image input operation by an image input section;

FIG. 3 shows an example of the contents stored in the input imagestoring section;

FIG. 4 is a block diagram showing the constitution of an imagediscriminating section;

FIG. 5 is a block diagram showing the constitution of an featuringamount calculation section shown in FIG. 4;

FIG. 6 schematically shows the projection by an entire image projectingsection shown in FIG. 5;

FIGS. 7A to 7D schematically show operations by an 8-adjacent linkedcomponents extraction section, a labeling/grouping section, and acoordinates calculation section shown in FIG. 5;

FIGS. 8A and 8B schematically show the operation by the 8-adjacentlinked components extraction section shown in FIG. 5;

FIG. 9 shows the featuring amount stored in the feature storing sectionshown in FIG. 4;

FIG. 10 is a block diagram showing the constitution of an image regiondiscrimination section shown in FIG. 4;

FIG. 11 shows the contents stored in a discrimination result storingsection shown in FIG. 4;

FIG. 12 schematically shows the discrimination result displayed by auser interface section shown in FIG. 1 and the specifying method of theimage region;

FIG. 13 shows an example of image region information which is stored inthe discrimination result storing section when the image region isspecified;

FIG. 14 schematically shows the methods of specifying and selection ofthe image region;

FIG. 15 schematically shows the selecting operation of selecting theimage region and a blank region;

FIG. 16 shows the constitution of a control panel and an interfacecontroller; and

FIG. 17 is a flow chart schematically representing the processingoperation of the image forming apparatus shown in FIG. 1.

BEST MODE OF CARRYING OUT THE INVENTION

An embodiment of the present invention will be described below withreference to the drawings.

FIG. 1 shows the constitution of an image forming apparatus according tothe present embodiment. The image forming apparatus comprises an imageinput section 1 constituted by such a device as a scanner to read animage on the original document and input into the apparatus; an inputimage storing section 2 for temporarily storing the image input by theimage input section 1; an image discriminating section 3 fordiscriminating the attribute of the input image by calculating afeaturing amount of the input image; an image region selecting section 4for selecting an image region in accordance with the user's instruction;an image processing section 5 for reading the image corresponding to theselected image region from the input image storing section 2 andprocessing the read image in accordance with the user's instruction; anoutput image data storing section 6 for temporarily storing theprocessed image; a user interface section 7 for displaying theinstruction of the user or the processing result to be informed to theuser; a controller 8 for controlling the user interface section 7; asystem control section 9 for integrally controlling the apparatus; asystem information storing section 10 for storing input information,processed information, or output information; and an image outputsection 11 for outputting the processed image. The apparatus may furthercomprises an external interface section 12 for connecting the apparatusto an arbitrary network. Via an arbitrary network such a public networkor as a LAN connected by the external interface section 12, for example,the image processed by the image processing section 5 can be output tothe external apparatus.

The above-mentioned section will be described below more specifically.

Image Input Section

The image input section 1 includes an input device such as a scanner,and is used to input image data into the image forming apparatus. Withuse of a scanner having resolution of 10 pixels/mm (8 bits/pixels) asthe input device for example, the information on the original document,which is to be input, is sampled to obtain 10 pixels for each 1-mmrange. (That is, the sampling is performed at the pitch of 0.1 mm.)

FIGS. 2A, 2B and 2C schematically show the information obtained by thesampling of the original document (e.g. the sampling of an originaldocument having A3 size [420 mm×297 mm] at a 0.1 mm pitch).

FIG. 2A shows 4200×2970 pixels obtained by the sampling. These pixelsare represented by about 12 megabytes of data if each pixel has 256tones. Assume that a character having a size of 4 mm×4 mm as shown inFIG. 2B is located at a distance of 40 mm in the X (lateral) directionfrom an origin (i.e., the upper-left corner of the original document)and at a distance of 40 mm in the Y (longitudinal) direction from theorigin. Then, as shown in FIG. 2C, the initial coordinates of thecharacter will be (400, 400), and the end coordinates thereof will be(439, 439), after the sampling. The image is sampled in units of an areaS as shown in FIG. 2C, and the resultant value is determined as thepixel value at the coordinates. For example, when the sampled value inan area located at a position 41.9 mm distant from the origin in thelongitudinal direction and 40.1 mm in the lateral direction is 220, thesampled value is determined as the pixel value at coordinates of (418,400).

FIG. 3 shows the relationship between the coordinates of the pixels andthe pixel values obtained by the sampling process.

FIGS. 2A, 2B, 2C and 3 show an example of the sampling operation withthe sampling frequency of 10 pixel/mm. The sampling with the othersampling frequency is performed also in accordance with the sameprinciple.

Input Image Storing Section

The input image storing section 2 stores the pixel values of the sampledpixels at addresses corresponding to the coordinates obtained by thesampling process, as shown in FIG. 3, for example. The pixel values eachrepresent the density of the pixel at the coordinates. In this example,the pixel value increases as the density of the pixel increases.

Image Discriminating Section

FIG. 4 shows an example of the constitution of the image discriminatingsection 3. The image discriminating section 3 comprises a featuringamount calculation section 3 a; a featuring amount storing section 3 b;an image region discrimination section 3 c; an image regiondiscrimination information storing section 3 d; and a discriminationresult storing section 3 e.

FIG. 5 shows an example of the constitution of the featuring amountcalculation section 3 a. A control section 31 controls the featuringamount calculation section 3 a during all the processing operation ofthe featuring amount calculation section 3 a. A featuring amount accesssection 32 generates an address for storing the featuring amountcalculated for the image of the original document in the featuringamount storing section 3 b shown in FIG. 4, a control signal and thelike. An image data access section 34 generates an address for readingthe image of the original document from the input image storing section2 shown in FIG. 1, a control signal, and the like.

A binarizing section 33 receives the image data read by the image dataaccess section 34 from the input image storing section 2 and binarizesit. The read pixel value is compared with the threshold value set inadvance. When the pixel value is larger than the threshold value, thepixel value is binarized as “1” (black point), otherwise, binarized as“0” (white point).

A binary image data memory 35 is a memory for storing the binarizedimage by the binarizing section 33.

An entire image projecting section 36 calculates longitudinal andlateral projections of an entire image on the basis of the binary imagedata stored in the binary image data memory 35. The projection valuesmaller than a predetermined threshold value is set as “0”, and theother projection value remains unchanged. “Projection” means the processof accumulating the number of only black pixels in each of the pixelcolumns and rows as the projection value of the column or row.

FIG. 6 shows an example of the entire projection obtained by the entireimage projecting section 36 on the basis of the binary image of theoriginal document data. By arbitrarily setting the threshold value (forexample, at 3% of the largest number of the pixels to be accumulated),various information in the binary image data of the original documentcan be obtained. FIG. 6 shows the case where the area to be read islarger than the original document area. By calculating the initialcoordinates [Tyi (i=1, 2, . . . , Ny) in the longitudinal direction, Txj(j=1, 2, . . . , Nx) in the lateral direction] of the hills of thelongitudinal/lateral projections and the widths [ΔTyi (i=1, 2, . . . ,Ny) in the longitudinal direction ΔTxj (j=1, 2, . . . , Nx) in thelateral direction] of the hills of the projections, various data in thebinary image data, such as the distances between the character columns,between the characters, and the margin surrounding the original documentarea can be obtained.

An example of the calculation method of calculating the originaldocument area on the basis of the scanning area will be described below.When the initial value of the scanning is (X0, Y0, where X0=0, andY0=0), and the end value is (Xm, Ym), the initial value (X′0, Y′0) andthe end value (X′m, Y′m) of the original document area are calculated.

When the initial coordinates Tyl of the first hill of the longitudinaldirection projection is “0”, the following relationship can be obtained:

Y′0=Y+ΔTy1.

When Ty≠0, Y′0 can be represented as

Y′0=Y0.

When the addition of the initial coordinates Tyn and the width ΔTyn ofthe last hill of the longitudinal direction projection is Ym, in short,when the relationship (Tyn+ΔTyn=Ym) can be obtained, Y′m can berepresented as shown below.

Y′m=Ym−ΔTyn.

In the case of (Tyn+ΔTyn≠Ym), Y′m is represented as Y′m=Ym.

X′0 and X′m of the lateral direction projection can be calculated in thesimilar manner. The scanning area extends from the coordinate (X0, Y0)to the coordinate (Xm, Ym). The area extending from the coordinate (X′0,Y′0) to the coordinate (X′m, Y′m) is the original document area. Whenthe scanning area and the original document area are different in size,the region (shaded area shown in FIG. 6) as the difference of the areasis regarded as an unnecessary region. The information such as the areainformation of the unnecessary region, the information of the startingand the end points of the scanning, and the information of the originaldocument area is stored in the image region discrimination informationstoring section 3 d.

A partial projection section 37 receives from the entire imageprojecting section 36 information of the initial points of the hills ofthe longitudinal/lateral projection, the widths of the hills, thescanning area, the original document area, and the like. On the basis ofthe information of hills of the longitudinal direction projection andthe widths thereof, the binary image data corresponding to each of thehills is read from the binary image data memory 35, and then the lateraldirection projection of each of the read image data is obtained.

By performing such a partial projection process, the number of the hills(objects) of the lateral direction projection and the initial point andthe size of each object are stored in the image region discriminationstoring section 3 d, so as to correspond the information of each of thehills of the longitudinal projection. The information of the lateralprojection such as an initial point MKZ of each object and the averagesize LHMS (average value of the width of the hills of the lateraldirection projection, which is generally regarded as the average valueof the character sizes) is sent to a coordinates extraction section 43.

Similarly to the partial projection section 37, the 8-adjacent linkedcomponents extraction section 38, a labeling/grouping section 40, andthe coordinates calculation section 43 calculate the coordinates of eachcharacter for specifying the position of the character for eachcharacter column. These sections execute a process basically similar tothat performed by the partial projection section 3, i.e., the extractionof characters. The pixels forming each character are not always linkedtogether. Thus, the characters are each extracted as a plurality ofobjects, merely by calculating the projection even if there is only onecharacter. These sections thus extract one character region bycalculating the linked pixel components on the basis of the positions ofthe pixels in more detail than by the process in the partial projectionsection 37.

The above-mentioned process by the 8-adjacent linked componentsextraction section 38, the labeling/grouping section 40, and thecoordinates calculation section 43 will be schematically described belowwith reference to FIG. 7. As shown in FIG. 7A, the process describedbelow is performed for binarized numerals “2” and “3” as binary imagedata to be processed. The binary image data as shown in FIG. 7A isfirstly subjected to a linked components extraction process of the8-adjacent linked components extraction section 38 in order to give alabel to the main pixel of the linked components, as shown in FIG. 7B.The labeling/grouping section 40 executes grouping of equivalent groupson the basis of the labels given to the pixels as shown in FIG. 7C. Onthe basis of the result of the grouping, the initial coordinates and theend coordinates of the linked pixels (i.e., the physical block) areobtained, as shown in FIG. 7D.

The 8-adjacent linked components extraction section 38 calculates thelinked components of the interest pixel. FIG. 8A shows adjacent cellsnecessary for obtaining the linked components of an interest pixel P, asan example. The linked components of the pixel P are calculated inaccordance with the following procedures:

When the pixel value of the pixel P is “0”, the label of the pixel P isset as “0”;

When the pixel value of the pixel P is “1” and all the labels ofadjacent pixels A, B, C and D which have been already processed are “0”,the pixel P is given a new label.

When the pixel value of the pixel P is “1” and at least one of adjacentpixels A, B, C and D has the label other than “0”, one (the minimumlabel, for example) of the labels other than “0” is selected to be setas the label of the pixel P, and it is recorded that all the pixelshaving labels other than “0” are equivalent.

Assuming that the coordinates of the pixel P to be processed is (7, 10)as shown in FIG. 7B, for example, and the adjacent pixels A, B and Cshown in FIG. 8A have been already processed and given label values asshown in FIG. 8B. The label value of the pixel to be processed is set at“1”, and thus the minimum one (i.e., “2”) of the label values other than“0” among the adjacent pixels is set as the label value of the pixel P.Further, there are pixels given the other label values than “0” (e.g.“2” or “3”) among the adjacent pixels, and it is thus recorded thatlabel value “2” is equivalent to label value “3”.

The label value of each pixel and the equivalent label information arestored in the linked components information memory 39.

The labeling/grouping section 40 reads label value (label data of thelinked component) of each pixel and equivalent label information(equivalent label data) from the linked components information memory39, groups the equivalent labels, and replaces the labels of the groupedpixels with new labels, as shown in FIG. 7C. The information of the newlabel added to the grouped pixels is stored in a labeling informationmemory 41.

Subsequently, the labeling/grouping section 40 scans the labelinginformation memory 41 to read the minimum coordinates and the maximumcoordinates which have the same label. The coordinates having the samelabel are contained in a physical block linked as one block, and thusthe minimum coordinates is set as the initial coordinates of thephysical block, and the maximum coordinates is set as the endcoordinates of the physical block. The information of these coordinatesis stored in a character coordinates extraction information memory 42.

The coordinates extraction section 43 executes unification of thephysical blocks stored in the character coordinates extractioninformation memory 42 on the basis of LHMS as the average size ofobjects and MKZ as the initial points of the objects which are set fromthe partial projection section 37. By performing this process,coordinates of one character constituted of a plurality of linkedcomponents for example, is calculated. In this example, a physical blocklocated in the area ranged from the initial point MKZ to the averagesize LHMS is regarded as one character. The coordinates of each of theunified physical blocks and the data thereof show that the minimumcoordinates is the initial coordinates of the character to be processed,and the maximum coordinates is the end coordinates of the character.

For example, assuming that

Physical block 1 having initial coordinates (xs1, ys1), and endcoordinates (xe1, ye1) and

Physical block 2 having an initial coordinates (xs2, ys2), and an endcoordinates (xe2, ye2) are unified, the initial coordinates (xs, ys) andthe end coordinates (xe, ye), and the size (Δx, Δy) of the unifiedphysical block are represented as follows:

xs=min (xs1, xs2)

ys=min (ys1, ys2)

xe=max (xe1, xe2)

ye=max (ye1, ye2)

Δx=xe−xs

Δy=ye−ys

The above-mentioned process is executed for all the character columns tocalculate the coordinates values or the sizes (Δx, Δy) of all thecharacters in the document. However, the physical block having theinitial point the end point of the scanning area or the initial pointthe end point of the original document area as its own coordinates valueis not unified with the other physical blocks.

In this manner, the coordinates extraction section 43 calculates thefeaturing amount. The featuring amount is written in the featuringamount storing section 3 b on the basis of a write address supplied fromthe featuring amount access section 32, in response to the instructionof the CPU. FIG. 9 shows one example of the contents of the featuringamount stored in the featuring amount storing section 3 b. The featuringamount storing section 3 b stores the coordinates of the initial pointand the size of every physical block (object) corresponding to onecharacter extracted from the coordinates extraction section 43, as thefeaturing amount of the physical block.

On the other hand, the featuring amount (the scanning area, the originaldocument area, the number of objects, the initial point, size, and thelike) extracted by the entire image projecting section 36 and thepartial projection section 37 is stored in the image regiondiscrimination information storing section 3 d.

The featuring amount storing section 3 b mainly stores the featuringamount of the image region in units of one character. The featuringamount stored in the image region discrimination information storingsection 3 d is more vague than that stored in the featuring amountstoring section 3 b.

Next, the image region discrimination section 3 c shown in FIG. 4 willbe described next. FIG. 10 shows the constitution of the image regiondiscrimination section 3 c. The featuring amount (initial coordinates,size, and the like) of each image object in the image of the originaldocument is supplied from the featuring amount storing section 3 b to afeaturing amount reading section 51. An image region discriminationinformation reading section 52 reads determination information aboutunnecessary region, scanning area, original document area, or the likefrom the image region discrimination information storing section 3 d.The section 52 reads the initial point, size of each object (character)from the partial projection section 37.

An attribute determination and layout analyzation section 53 determinesthe attribute (character region, photograph region, or the like) of eachof the objects on the basis of the information supplied from thefeaturing amount reading section 51 and the image region discriminationinformation reading section 52. Further, the attribute determination andlayout analyzation section 53 unites a collection of the objects havingthe same attribute so as to discriminate one or a plurality of imageregions (each consisted of one or a plurality of objects) from the imageof the original document.

The collection of the objects can also be united by extending the resultof the longitudinal projection and the result of the lateral projection(shown in FIG. 6), for example, for a predetermined distance, in thelongitudinal direction and the lateral direction, respectively. Theresult of the discrimination is stored in the discrimination resultstoring section 3 e, as is shown in FIG. 11.

The word “attribute” means a type of an image region, e.g., a characterregion or a photograph region. Hereinafter, the region other than imageregions given such attributes present in the original document, i.e.,the image region containing nothing will be called “blank region.” Thearea of the blank region may be stored as one of the attributes of theimage region, in the discrimination result storing section 3 e.

FIG. 11 shows an example of the contents of the region discriminationresult stored in the discrimination result storing section 3 e. Thesection 3 e stores the number of the discriminated image regions, thecoordinates of the initial point of each image region, the size of eachimage region, the attribute given to each image region. Further, thesection 3 e stores the data showing whether or not the user hasspecified some points, as will be described later.

Image Region Selecting Section

The image region selecting section 4 provides the discrimination resultin the original document image region discriminated by the imagediscriminating section 3. For example, as shown in FIG. 12, the imageregion selecting section 4 sends to the user interface section 7 thepositions of the four image regions A, B, C and D which arediscriminated from the image of the original document.

The discrimination result of the original document image region by theimage discriminating section 3 may be displayed in a form of the contourof the image region in the frame of the original document, as shown inFIG. 12, on the basis of the positional information of the image region,which is extracted by the attribute determination and layout analyzationsection 53 shown in FIG. 10. Further, attribute information such ascharacter region or photograph region or the like may be simultaneouslyindicated by the display (in the contour of each image region, forexample). Otherwise, the contour indicating the area of the extractedimage region may be overlapped on the image of the original document.Further, the attribute, area, or the like of each of the extracted imageregions may be indicated by character information, as shown in FIG. 11.

When the user specifies a point P1 in an image region which the userwants to process on the basis of the image region discrimination resultof the image of the original document indicated by the user interfacesection 7, the positional information of the point P1 is input into thesystem control section 9 via the interface controller 8. The systemcontrol section 9 stores the positional information as the specifiedcoordinate value (Xu, Yu) in the system information storing section 10,and also supplies to the image region selecting section 4.

The image region selecting section 4 determines to which image regionthe specified position belongs, on the basis of the specified coordinatevalue (Xu, Yu) and the positional information of each image region asthe discrimination result by the image discriminating section 3. Theimage region selecting section 4 sets in the discrimination resultstoring section 3 e of the image discriminating section 3 a flag (“1”)of the specified image region, which indicates whether or not the imageregion is specified by the user, as shown in FIG. 13. FIG. 13 shows thecase where the region B is specified.

An example of a method of determining the image region corresponding tothe point (Xu, Yu) specified by the user on the basis of the specifiedpoint will be described below.

If the following conditions are satisfied, the specified coordinates islocated in one of the image regions in the image of the originaldocument (see P1 in FIG. 12),

[(Yis≦Yu) and (Xis≦Xu)] and

[(Yis+ΔYi)≧Yu] and [(Xis+ΔXi)≧Xu]

where the initial point of each image region in the image of theoriginal document is (Xis, Yis ), and the size is (ΔXi, ΔYi).

The image region including the specified coordinates is thus determinedas the specified region. The specified region is displayed in brightnessdifferent from that of the other regions and is thereby distinguishedfrom the other regions.

When the inner point in one of the image regions in the image of theoriginal document cannot be specified due to the function, size, or theinput device of the display of the user interface section 7, the imageregion nearest to the specified point may be selected instead thereof.

As shown in FIG. 14, a plurality of image regions A, B, C and Dextracted from the image of the original document are indicated by theuser interface section 7. When a point P2 positioned between the imageregions A and B, is specified by the user, since the image region A isnearer to P2 than the image region B, the image region A is selected.

Where the coordinates of P2 is (Xp2, Yp2), the image region A has theinitial point (XaS, YaS) and the size (ΔXa, ΔYa), the image region B hasthe initial point (XbS, YbS) and the size (ΔXb, ΔYb), the image region Chas the initial point (XcS, YcS) and the size (ΔXc, ΔYc), and imageregion D has the initial point (XdS, YdS) and the size (ΔXd, ΔYd), thenthe end point of the image region A is represented as (XaS+ΔXa=XaE,Yas+ΔYa=YaE), the end point of the image region B is represented as(XbS+Δxb=XbE, YbS+ΔYb=YbE), the end point of the image region C isrepresented as (XcS+ΔXc=XcE, YcS+ΔYc=YcE), and the end point of theimage region D is represented as (XdS+ΔXd=XdE, YdS+ΔYd=YdE). Therelationship between the points are represented as follows:

(YaS, YbS)<Yp2

(YaE, YbE)>Yp2

(YcS, YdS)<Yp2

On the basis of the above, the point P2 can be determined to be locatedbetween the image regions A and B.

Further, |(XaE−Xp2)| is the minimum value among the absolute values|(XaS−Xp2)|, |(XaE−Xp2)|, |(XbS−Xp2)|, |(XbE−Xp2)| where |X| representsthe absolute value of X. The image region A is thus determined to be thenearest to P2, and then the image region A is selected.

When the user specifies a point P3 positioned between the image regionsB and D, the image region D nearer to P3 than the image region B isselected.

The coordinates (Xp3, Yp3) of P3 can be represented as follows:

(XbS, XdS)<Xp3

(XbE, XdE)>Xp3

(XaE, XcE)<Xp3

On the basis of the above-presented relationship, P3 can be determinedto be positioned between the image regions B and D.

The absolute value |(YdS−Yp3)| is the minimum one among the absolutevalues |(YbS−Yp3)|, |(YbE−Yp3)|, |(YdS−Yp3)|, |(YdE−Yp3)|, and the imageregion D is determined to be nearest to P3 and then selected.

Similarly, when a point P4 is specified, the image region C is selectedsince the image region C is the nearest to P4 among the image regions A,B, C and D.

The process of selecting an image region containing, as an attribute,characters or photographs, by specifying one point on the image of theoriginal document has been described. The present invention is notlimited to this process. A non-image region, or a blank region, may beselected from the images of the original document on the basis of thespecified coordinates values, when the specified coordinates are notpresent in the extracted image region.

One example of the methods of selecting an image region in the lattercase will be described below.

This method of selecting a blank region helps to insert, in the originaldocument, a desired image into a blank region which contains nothing.

Assume that a plurality of image regions A, B, C, and D extracted fromthe image of the original document is displayed in the user interfacesection 7, as is illustrated in FIG. 15. When a point P2 is specified bythe user, the blank region w1 existing between the image regions A and Bis selected, because the point P2 exists in the blank region w1. Theinitial point (Xp2S, Yp2S) and the end point (Xp2E, Yp2E) of therectangular blank region w1 are calculated as shown below:

Xp2S=XaE+ΔMx

Xp2E=XbS−ΔMx

Yp2S=max (YaS, YbS)+ΔMy

Yp2S=min (YaE, YbE)−ΔMy

where max (i, j) is a function for selecting the maximum one of thevalues i and j, and min (i, j) is a function for selecting the minimumone of the values i and j. ΔMx and ΔMy are respectively offset values inX and Y directions, and default values may be set by the system, or setby the user.

Similarly, when P3 is specified, the blank region w2 is selected,because P3 is located in a blank region w2 present between the imageregions B and D. For example, the initial point (Xp3S, Yp3S) and the endpoint (Xp3E, Yp3E) of the rectangular blank region w2 are calculated bythe process as shown below.

Xp3S=max (XaS, XdS)+ΔMx

Xp3E=min (XbE, XdE)−ΔMX

Yp3S=YbE+ΔMy

Yp3S=YdS−ΔMy

P4 is not located in the region held by any two of the image regions A,B, C and D, unlike the cases of P2 and P3. According to theabove-mentioned method, when P4 is specified, two blank regions w3 andw4 are extracted.

The Blank region w3 has the coordinates as shown below:

Xp4S=min (XaS, XcS)+ΔMx

Xp4E=max (XbE, XdE)−ΔMx

Yp4S=max (YaE, XbE)+ΔMy

Yp4E=min (YcS, YdS)−ΔMy

The Blank region w4 has the coordinates as shown below:

Xp4S=max (XaE, XcE)+ΔMx

Xp4E=min (XbS, XdS)−ΔMx

Yp4S=max (YaS, XbS)+ΔMy

Yp4E=min (YcE, YdE)−ΔMy

As described above, more than one region may be selected depending onthe position of the specified point. One of these image regions thusspecified and selected by the user may be automatically selected as aregion in which another image will be inserted, if this image region hasdesired features, such as a desired size. Alternatively, the imageregions thus selected may be displayed to the user through the userinterface section 7, so that the user may select one of them which isdesirable to him or her.

User Interface Section

The user interface section 7 can be constituted of various I/O devicessuch as an LCD indicator (control panel) comprising a key board, a CRT,a touch panel, and the like. A user interface constituted of a controlpanel will be described below. The user interface according to thepresent invention is not limited to that comprising a control panel, butthe other I/O devices may be used without departing from the spirit andscope of the invention.

FIG. 16 shows the constitutions of the user interface section 7 and theinterface controller 8. The user interface section 7 is mainlyconstituted of a control panel substrate 7 a, and the interfacecontroller 8 is mainly constituted of a control substrate 8 a.

The control panel substrate 7 a is provided with an LCD indicator (LCD)75, a touch panel, and various operation keys 77, and a control keymicrocomputer 78 for checking the depression state of the I/O devices.The control key microcomputer 78 sends the depression state of the I/Odevices to a microcomputer 82 of the control substrate 8 a. Themicrocomputer 82 processes information sent from the control keymicrocomputer 78, information from various sensors such as a sheetsensor 92 connected to the control substrate 8 a and a tonerconcentration sensor 95, in accordance with various control programsstored in a ROM 80 and a RAM 79, and the process result is displayed inthe LCD 75 by the LCD controller 81.

The ROM 80 stores data to be indicated in the display, the program data,or the like, and the RAM 79 stores a period of time required for theswitching the display. The RAM 79 further store information necessaryfor indicating the image regions sent from the image region selectingsection 4, or the other parameters, for example. The image regioninformation is then sent to the system information storing section 10through the system control section 9 of FIG. 1.

The information necessary for displaying the image regions extractedfrom the image of the original document is controlled by themicrocomputer 82 to be read from the RAM 79. On the basis of the readinformation, the display data is prepared. The display data is sent tothe LCD 75 through the LCD controller 81, and displayed as shown in FIG.12.

The control substrate 8 a mainly controls the LCD 75, the touch panel77, and the like, and controls the motors to drive, in necessity. Inaddition to the above-mentioned functions, the control substrate 8 adetects sheet jam, empty toner cartridge, and the like, in receiving asignal generated by the sheet sensor, a waste toner sensor, a tonerconcentration sensor, or the like, and buzzes a buzzer 76 presented onthe control panel, if necessary.

The LCD controller 81 sends a display control signal to the LCD 75 onthe basis of the signal sent from the microcomputer 82, the graphicdisplay is thereby attained thereby.

Image Processing Section

The image processing section 5 performs the process specified by theuser for the image region selected by the image region selecting section4. The following is one example of the image processing:

1. Image Editor:

There may be various methods of editing images. In one of these methods,a selected image region can be clipped to be magnified. Themagnification may be performed, for example, by means of linearinterpolation which is generally used.

The image processing section may have editing functions (digital imageprocessing) generally provided in the conventional word processors orthe like. The editing functions are changing the color (e.g., black tored) of the selected region, and changing the characters in the regionto outline or slanted ones.

2. Image Replacement:

The image replacement is a process of replacing a selected region withthe other image data input lately. For example, it is determined whetherthe data to be inserted should be magnified or reduced, on the basis ofthe initial and end coordinates of the image region and the data to beinserted. If the data should be magnified, it is magnified and insertedinto the region, instead of the image present in the region. If the datashould be reduced, it is reduced and inserted, instead of the imagepresent in the region. If the data need not be magnified or reduced, itis inserted, without being changed in size.

3. Image Insertion:

According to the apparatus of step 1, secret or key information can beinserted in an invisible state into the selected image region. Theinformation insertion process is not performed on the entire originaldocument, but in a limited region specified by the user. The informationinsertion process is, however not the main feature of the presentinvention, and thus the detailed description thereof will be omitted.

When the specified region is a blank region, not only the invisibleinformation but also a visible image can be inserted. When a visibleimage is inserted into the blank region, the image is magnified/reducedon the basis of the area of the blank region and the size of theinformation to be inserted, and then inserted therein.

The user may select desired one of the above-mentioned processes withuse of the user interface section 7. The entire image of the originaldocument which includes the image in the image region processed by theimage processing section 5 is temporarily stored in the output imagedata storing section 6.

Output Image Data Storing Section

The output image data storing section 6 is intended to temporarily storethe image output through the image output section 11. When a pluralityof images is united into one piece of image information, the outputimage data storing section 6 needs to function as a page memory. Whensome information needs to be deleted from one image of the originaldocument, the output image data storing section 6 needs to function as aline buffer of only few lines.

Image Output Section

The image output section 11 outputs the image stored in the output imagedata storing section 6. The image output section 11 is constituted of,for example a printer which outputs the data printed on a sheet.

System Control Section

The system control section 9 controls all the operations related to thepresent invention by the system. The system control section 9 isconstituted of a CPU and the other peripheral devices. The systemcontrol section 9 supplies a timing signal, a clock signal, a controlsignal, and parameters used for various processes to the above-mentionedsection.

System Information Storing Section

The system information storing section 10 stores data or programnecessary for the entire system.

External Interface Section

The external interface section 12 is provided to send to the externalapparatus through an arbitrary communication network the image regionprocessed by the image processing section 5 or the entire image of theoriginal document which includes the specified image region. Theexternal interface section 12 is, for example a FAX MODEM using atelephone network, an interface for connecting the apparatus tospecified network.

When the system comprises the image forming apparatus according to thepresent embodiment is connected to a plurality of image formingapparatuses through the external interface section 12 and a network, thesystem becomes sufficiently effective if at least one of the imageforming apparatuses comprises an image output section 11. Morespecifically, the image of the selected region can be processed,transmitted through the external interface section 12 and printed, evenif the region has been selected by an image forming apparatus (or imageprocessing apparatus) that does not have the image output section 11.

Process Operation

The following is the description of the process operation by the imageforming apparatus having the above-mentioned constitution, withreference to the flow chart shown in FIG. 17.

At first, the image of the original document which is to be processed isinput into the image input section, and temporarily stored in the inputimage storing section 2 (STEP S1). The image of the original documentstored in the image storing section 2 is discriminated by the imagediscriminating section 3 (STEP S2), and the discrimination result isdisplayed by the user interface section 7. In this time, image regionsdiscriminated from the image of the original document are also displayed(STEPS S3 and S4). While watching the display, the user specifies atleast one point (STEP S5). The image region selecting section 4 selectsthe image region corresponding to the coordinates of the specifiedpoint. The selected image region is displayed to the user by the userinterface section 7 (STEP S6). If the user interface section 7 has somekeys to input such instructions such as “confirm” or “cancel” theselected image region, the following process is performed in accordancewith the instruction input by operating the keys (STEP S7). Morespecifically, when the key of “confirm” is depressed (“YES”), a desiredprocess is selected to be performed for the selected image region(“EXIT”). On the other hand, when the key of “cancel” is depressed(“NO”), the process backs to STEP S3 to repeat the above-mentionedsteps.

As described above, according to the above-mentioned embodiment, adesired image region in the image of the original document can be easily(just by user's one action) specified with use of image discriminationtechnique, and the specified image region can be displayed. When thespecified image region needs to be corrected, the specified image regioncan be corrected after specifying the portion to be corrected isspecified and displaying the image constitution after the correction.

What is claimed is:
 1. An image processing apparatus comprising: imageregion determination means for determining a plurality of image regionsin the original document on the basis of input image data of theoriginal document; display means for displaying positions of the imageregions determined by the image region determination means in the imageof the original document; specifying means for specifying at least oneposition of a point on the basis of contents displayed by display means;determination means for determining an image region corresponding to thespecified position as a specified region on the basis of coordinates ofthe position specified by the specifying means and relative positions ofthe image regions, and for determining a non-image region as a specifiedregion when the specified position is located in the non-image regionwhich lies between at least two regions that are determined by the imageregion determination means, wherein said display means displays thenon-image region by discriminating the non-image region from the otherimage regions; and image processing means for processing an image in thespecified region determined by the determination means.
 2. The apparatusaccording to claim 1, wherein the display means has means for displayinga contour of each of the image regions determined by the image regiondetermination means at a corresponding position in a frame of the imageof the original document.
 3. The apparatus according to claim 1, whereinthe image region determination means comprises; featuring amountcalculating means for calculating a position and a size of each ofobjects constituting the image of the original document in order tosupply them as a featuring amount; and means for determining a pluralityof image regions by unifying each of collections of the objects havingthe same attribute on the basis of the featuring amount.
 4. Theapparatus according to claim 1, wherein the image processing means hasimage insertion means for inserting another image into the image of thespecified region.
 5. An image forming apparatus comprising: scanningmeans for optically scanning an image of an original document to supplyimage data of the original document; image region determination meansfor determining a plurality of image regions in the image of theoriginal document on the basis of the image data of the originaldocument supplied from the scanning means; display means for displayingpositions of the image regions determined by the image regiondetermination means in the image of the original document; specifyingmeans for specifying at least one position of a point on the basis ofthe positions displayed by the display means; determination means fordetermining as a specified region an image region corresponding to thespecified position on the basis of coordinates of the position specifiedby the specifying means and relative positions of the image regions, andfor determining a non-image region as a specified region when thespecified position is located in the non-image region which lies betweenat least two regions that are determined by the image regiondetermination means, wherein said display means displays the non-imageregion by discriminating the non-image region from the other imageregions; image processing means for processing the image of thespecified region determined by the determination means; and imageforming means for forming an image on the basis of the image data of theoriginal document which is supplied from the scanning means and theimage data processed by the image processing means.
 6. The apparatusaccording to claim 5, wherein the display means has means for displayinga contour of each of the image regions determined by the image regiondetermination means at a corresponding position in a frame of the imageof the original document.
 7. The apparatus according to claim 5, whereinthe image region determination means has a featuring amount calculatingmeans for calculating position and size of each object constituting theimage of the original document so as to provide the position and thesize as a featuring amount, and means for determining a plurality ofimage region by unifying each of collections of objects having the sameattribute respectively on the basis of the featuring amount.
 8. A methodof specifying image regions, comprising: image region discriminationstep of discriminating a plurality of image regions in an image of theoriginal document on the basis of input image data of the originaldocument, display step of displaying a position of the image regionsdiscriminated in the image region discrimination step in the image ofthe original document, specifying step of specifying at least oneposition of a point on the basis of the position displayed by thedisplay step, and determining step of determining an image regioncorresponding to the specified position as a specified region on thebasis of coordinates of the position specified in the specifying stepand relative positions of the image regions, and of determining anon-image region as a specified region when the specified position islocated in the non-image region which lies between at least two regionsthat are discriminated by the image region discrimination step anddisplaying the non-image region by discriminating the non-image regionfrom the other image regions.
 9. The method according to claim 8,wherein the display step has a step of displaying a contour of each ofthe image regions discriminated in the image region discrimination stepat a corresponding position in a frame of the image of the originaldocument.
 10. The method according to claim 8, wherein the image regiondiscrimination step has a featuring amount calculation step ofcalculating a position and a size of each of objects constituting theimage of the original document so as to supply the position and the sizeas a featuring amount, and a step of discriminating a plurality of imageregion by unifying collections of the objects having the same attributerespectively on the basis of the featuring amount.